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This paps, s—zas fop, separate projects recettdy completed I or it at die MIT Mar-Machine Systetns 
Laboratory. Four others are descnbed in a companion paper in Volume . 

1. OPERATOR ADJUSTABLE IMPEDANCE IN MASTER SLAVE TELEOPERATION - 

G. Jagganath Raju 

Abstract. A 2-port impedance network modelof presented, 

a human operator at the master port interacts wi J for the servomechanisms that transmit motion 

■“SldSZ operator ' ^ttl.btg design gnidelh.es gntnan.ee stability for any passive 
task object at the slave port and any passive human impedance at the master port. 

improve ^^J^'^J^^jJ^^^g^^^y^^^theh'enyironntent, two ^^^*Jj?^ I ^^^^^l s U ^^, a (b e V ,jbjeebve 
thp "muscle senses” that mediate kinesthesis and proprioception. The assumption made h 

of the manipulation task is to identify and/or alter the location of “ b^yTuUn sensation to the remote 

"telepresence” reflects the concept of transporting e iim pe mechanism and/or the telecommunication 

IndlSdogrSnsJr r'etiuUoMr^smission bandwiSh, time delays) the transmission signals are degraded and 
have to be enhanced or compensated for tn some way to be -of r ^.^ a ^ f ^ force i eve ls i n the interaction with 

actuator to redirect a sense of teel to tne opera . manipulate the 

transmits the forces that the human operator would have imposed on the task had she been torn P 

object directly. , , .. w cm a<; a 2 oort network with the operator-master arm interface 



Figure 1. Remote manipulation. 
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The three sub-systems are: the human operator, who manipulates the master device of the MSM in a 
manner that results in the slave device of the MSM acting on the task to achieve the desired goal; a MSM which 
ansmits forces and motion between the human operator and the remote task; the task object in the remote 
environment that is being manipulated by the slave device of the MSM. 

Sum* CO “ St . itutCS an "ideal” telemanipulator? Intuitively a reasonable response would be: An ideal 

talk^STw t0r ll 0 " 6 ? at i > ^'? eS < ;° mplete trans P arenc y of the interface. In other words the operator feels as if the 
k bei '?- 8 ha ? d,ed direct| y- For force-feedback systems Handlykken [ 1 ] suggests that this can be 

™ ^ “ m ‘ y Shff 3nd WeightIess mec hanical connection between the end-effector of the master arm and 

th, no«t.Z°.S?° ji “ d Y ° Shil Tf [2] ^ ** ideal res P° nse of a remote manipulator system is one in which 

the position and force responses of the master arm are systematically equal to the responses of position and force 

when the operator manipulates the object directly. But from a human factors point of view, v£tut [3] suggests that 
the operator may sometimes get tired of holding a constant weight, and reports implementing a system wS 

Sthehiph ' "J? ,0n , °f? e rati ° t0 reduCe fatigue and “"Prove precision. IndLl if is not all that clear 

that the highest level of force-feedback is universally beneficial in executing all tasks. 

. , /I* 11 4 J r j POrt f **??* , in the cIassic peg-in-hole insertion task, the insertion phase shows little difference with 

or without force feedbrck Bejczy and Handlykken [5] report from experimental studies that there seems to be an 
optimal combination of the force-feedback gain (from slave to master) and the feedforward gain (from master to 
slave) and that this combination may be task dependent. 

iHuST" that “ no consensus regarding an universal idealization of a remote manipulator 

system. Indeed to some extent the hypothesis m this work is partly motivated by the non-existence of such a 

t stand 7 1 ’ s,n “*^ s brings out^the necessity of designing an adjustable system. An additional implication 
of this statement may be that the ideal” telemanipulator is an "adjustable” one. 

* A ** CinpUt °5 master P° rt » the MSM interacts with the operator; at the output or slave port it interacts 
with the task object m the remote environment, as illustrated in Fig. 2. 
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Figure 2. Electrical network model of master-slave manipulator. 


Electrical network model of master-slave manipulator. At each port (master and slave) the co-energy 
variables of interest are the effort variables (torques T m and T s ) and the flow variables (velocities Ci m and CL). 
Either of the co-energy variables at each port may be chosen to be the independent variable, and the value of the 
dependent variable is then determined by the parameters that characterize the MSM. If the flow variables are 
considered to be the inputs, the MSM can be represented by an impedance matrix [Z(s)J such that 
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Two other represenddions of the MSM are obtain^ one o f the 

at the other port is a flow variable, or vice-versa. e hvsica l svs tem the elements of each matrix can be 

relevant to our context are the bilateral 2-port an e . non . zero signal flow takes place in both 

two off-diagonal elements are ^^JCSble for any passive termination Z h (s) at the master 

in the RH^ (sTMesTzt! <j») are simple, and dt. hermitian residue matrix .teach of these poles.s 

apparatuses, experiment was conducted tocheck 'ZSSZZZt* 

SSSUSSJ 5SS5S if mher crit.n.soch 
,hebes '' 

tasks that have differing characteristics. Be ore con ac , but not re sult in the imposition of excessive force 

and the master port impedance high, so that con c should be increased so that the task can be executed, and 

on the task object. Upon contact the slave port im^ adequate level of force-feedback to the operator. The 

“ dCd^«-^ » <*« charaeterisdcsof the opeimor and die tash is an are, that can 

E* explored father with the aid of design tools that have been developed in this work. 

ofSen, M. andTunter, T„ Condo, system analysis and synthesis for. six DOF universal, fine. WEE 

«rJiS! 7 Fora FM SRI 

Controller Proceedings of the 17th Annual Conference on Manual Control UCLA ^f e,es ’ CA > 

MA, Sept. 1988. 
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RJSrSZ 1 *' 06 T,ME DELAYS: A PREDICTIVE aid with force 


Abstract. Delays in teleoperator control loops make "real time” control difficult. Operators tend to adopt a move- 
an -wait strategy (Ferrell, 1965, 1966). Noyes and Sheridan found that a "predictor display” which showed the 
’AT” 1 ** ‘1** SaVC t0 * e , master commands helped the operator work faster and more continuously. 

co " eq » ,0 inclutk fTO ***** of th. 

^£l?^ per !lf 0r Aid ' manipulator commands are sent through a model of the manipulator system 
without the delay, the response of the model should predict the delayed output which we will receive from the actual 

imuuier Ve ^ Ch ° 1CC ° f 051118 *** rCa1 ’ de,ayed output or ^ P redicted (model’s) output; or we can 

a ct..-.i 3 S / h ? WS ^ 8« ner al system diagram for the open loop predictor (below) running in parallel with the 

actual manipulator (above). The delay is shown in the feedback. It can be placed in either or spht between them 

with no loss of generality. Also, keep in mind that the forward loop gain, K\ as well as the preceding summing 
junction exist within the human operator. F S summing 


r(a) 




Figure 3. General system diagram for open loop predictor. 


j . U J? “f °? y * ereal feedback ^ no attenuating filtering (H(s)=l, Hp(s)=0), we observe the standard 
i me delayed f eedback system with its well known stability problems. However, using only the predicted feedback 

(H(s)- 0, Hp(s)=l) we no longer have to worry about the delay induced instability, but we lose accuracy and 
disturbance robustness. 


Thf predictor display has the effect of giving the operator visual feedback from both the real manipulator 
and the predictor. The operator then subconsciously combines them to get his "best estimate”. Since the normal 
means of receiving force feedback is strongly coupled to the position control input, operators seem to have more 
trouble using delayed force feedback than delayed video. Delayed force feedback tends to look like a disturbance. We 
want to find the best combination of actual and/or predicted force feedback for the operator. The predictor display 
serves as a model. 


In the most directly analogous form we provide both the delayed feedback and the predicted feedback to the 
operator. The predicted feedback is applied directly to the backdrive input joystick. The delayed feedback is applied 
to a passive joystick; this separates it more from the loop, letting the operator use it as desired. This presentation 
form is called "dual” force feedback. 
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em,, buying the priced feedback fo, the higher frequencies lessens 

the stability problem. 

Experiments. Testing of operator performance using force fcedba^dn^other for^aMi«!hid^ed W * 0m 

tasks: grappling of a floating mass and insertion of the mass into a slot. 

“'VP., Remote Manipulation with Transmission Delay, IEEE Transactions On Human Factors in 
FflZZu'w R ‘mSfo™ Feedback, Human Factors, October 1966, pp. 449-455. 

® ST. ANovel Predictor for Telemanipulation through a Time Delay, Proceedings of 

2 &52££E!£ L iiMM Moffett Field, CA: NASA Ames Researoh Center. 


MODELING AND COMMUNICATING OF INTENTION TO A COMPUTER 


. Wael Yared 


3. 

Abstract A system has been constricted to infer intent from the operator’s commands and the teleoperation 
context and generalize this information to interpret future commands 

pragmatics. ^ ^ ^ „ change „ ^ Wonment ». Inter faces such as robot command languages have 

i sssassfe 

aggrayatingmteofTMkii^ttm purpcro’mdin£eiM0deF4hJ^^“^^f^fN^ c . enc y and robustness ofhuman- 

task in the cument environment; the human is assumed to beknowledgeable and fully cooperative. 

Intention recognition. The purpose of the recognition algorithm is not to check the user’s beliefs and 

plans The following subsections correspond to the two main layers of the algorithm: the intention recogm 

algorithm proper, rnd ^ , of pHmi.ivo actions ^mp^t some 

task in the simulated environment (e.g. retrieving a block from a warehouse row switching blocks etc.). As 
bsUctfons Ttyped, the system parses them and ’’interprets” them by properly directing pointers to the 
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corresponding executable code. At that stage the task could, in principle, be carried out by the system in the 
environment without any further user intervention. This is where the intention recognition routine enters the game, 
however. Instead of giving the user the option of having the task executed, the system proceeds to reason about the 
plan just defined to elicit the intentions behind it. 

What the system does, in effect, is to simulate for itself the effects of that sequence of actions 
incrementally. To construct this chain of simulated effects, the system has to build a snapshot of its state vector and 
of the world model after applying each action, and update its private copy of both. Since the chain of actions has 
already been provided by the user, it is a simple matter to test, at each step in the plan, whether the following action 
is already executable or not. The system thus identifies the intended acts from the enabling acts in the plan, and 
stores this information for later use. Upon completion of that stage, the plan is appended to a dynamic plan library. 
The following points must be emphasized: 

(a) The cost of this exercise is negligible: the sequence of instructions is to be entered by the user anyway, 
and the procedural code for the primitive actions already exists- whether intention recognition is done, or whether 
straightforward ”dumb” plan execution is desired. Any robot command language has to include at least these, and 
nothing else is required to perform intention recognition. The computational cost of testing the necessary conditions 
for enablement is proportional to the size of the plan and thus poses no problem. Testing for the sufficient 
conditions could, in theory, be more problematic; I haven’t looked at a worst-case scenario yet. 

(b) It is not assumed that each act enumerated by the user is an intended act; rather, they are all intentional 
acts. The distinction is not futile, since only intended acts are the ones the user is committed to both in terms of 
reasoning (planning) and in guiding his or her behavior according to the intended act’s success or failure. 

(c) In contrast to work in planning, no action interrelationships are posited from the start. The system is 
given a linear, unstructured chain of acts and proceeds to infer nondeductively its corresponding structure. 

(d) The intention recognition process is invisible to the user; the intention recognition routine doesn’t, in 
any conventional sense, belong to the user-interface properly speaking, nor to the execution monitor or top-level 
control of a planner. Although no psychological relevance is claimed in any part of the present work, it is useful to 
take note of this in trying to dispel the conventional ’’clean” separation between ”smart” planner and ”dumb” top- 
level control. 

Plan re-interpretation. A plan for a physical manipulation task is re- interpreted if that task is invoked by the 
user in some different context than the one in which it was originally defined. When this case arises, the plan is 
reevaluated and eventually modified; if modified, it is appended again to the plan library. The plan re-intepretation 
procedure comprises the following steps: 

(a) The system locates in the plan library the most recent precedent of the plan definition, together with the 
latter’s corresponding model of the world environment at the time of the original definition. If the plan definition 
includes parameters, the system performs the trivial task of substituting the current parameters for the old ones in the 
original plan definition. At this point the plan is re-compiled, and the system proceeds to the relevance check. 

(b) In a first phase of the relevance check, the system performs a backward pass on all the subgoals of the 
modified plan in order to determine which of them are still relevant. A subgoal is no longer relevant if the current 
state of the world and the robot are identical to the ones it is there to achieve. The state of the world and of the robot 
a subgoal achieves is calculated by aggregating the effects of all the previous subgoals with those of the intended act 
of the current subgoal. Since it appears reasonable to assume that when a subgoal is irrelevant, then all its preceding 
siblings become irrelevant, the backward pass is stopped at the first occurrence of an irrelevant subgoal. Pointers are 
then redirected to ensure that the new plan only comprises the relevant subgoals. 

(c) In a second phase of the relevance check, the system checks the relevance of the enabling acts for each 
(now relevant) subgoal. Within each subgoal, a backward pass evaluates the relevance of the enabling acts with the 
same mechanism that was used in the intention recognition routine. In other words, the executability of each 
enabling act is checked in turn (in the context of the current subgoal); when an enabling act is found executable, its 
preceding siblings are all no longer relevant and are discarded from the plan. 

Implementation. The ideas presented in the previous sections have been implemented in an experimental robot 
interface called GRICE (Generalization through Recognition of Intention and Chains of Enablement). GRICE 
simulates a 2-D cartesian manipulator in a task environment that includes a warehouse and several blocks. The user 
configures the environment (places the blocks and the robot) with the mouse, or retrieves a pre-configured one from a 
data file. A menu then prompts the user to define a new task, retrieve a previously defined task, generalize on a 
previously defined task, or execute a current task. 

GRICE is implemented in the C language on a Silicon Graphics IRIS 2400 system. The reasoning 
routines, the task execution routines, and all the data structures (about 50% of the 3000 lines of code) are totally 
machine-independent and domain- indepedent. Some of the domain-dependent routines (the repertoire of robot 


86 



primitives, but no. rite command interpreter) end dm gmphic simulation dmwheavily on the IRIS graphics librrny 
^ ^'^''Rrtas^mh^s'for^ans (bodt source and ' > i naI ^’ , * r ^|^ s ^^^^^^^^ X /^ < ^jecbori«it«l' 

C, this is implemented using pointers to functions. 

4, COMMAND LANGUAGE FOR OBSTACLE AVOIDANCE -JongPa.lt 

language context is finding a collision-free path for a robot. 

Background and needs. A complete Us: k-jevel robot 

too long to be used on-line. . . k in ^ notion s of "supervisor control”, it is 

In order to overcome this difficulty, P g i n u a i decisions letting the computer 

suggested that a human operator should play a operator should decide how to maneuver to 

deal with the details. In the context of collision avo, ^,^ ^uTa clmand language should be reliable, 

s= easy and natural for the operator to use, and should be easy to 

learn. 

Command language. In dm command language be** »tjm£ the “^^Srection 

commands to a keyboard (a symbolic input) and simul ^^ ly (tQ Object or attributes of an object such 

or magnitude (an analogic input). He may also “° v * ^ calling their names) in coordination 

° P ' ra '" Bod, syntactics of dm command hutguage (eg., die ^^^^Sir'Sb'S^ 
what is admissible) and the semantics (specification of motions and l^aUon ^ ^ approach B until x 

to the environment) are important. In Figure , or exmnp , l^elv to be more relevant to what matters to the 

away ” than ”A go y along normal to toL 1 inch away from (this) edge in 

operator than the actual distance travelled. Spec yi g . \ iiirelv to be easier than saying 

«•* -go 3 .bo™ «m ***-. 


x 



Figure 4. A approaches B. 
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Hardware and software. The system hardware consists of three input devices (keyboard, mouse, 6 degree-of- 
freedom joystick) and graphics workstation. The software consists of six modules as shown in Figure 5: natural 
language interface (NU), indication interpreter (II), general interpreter (GI), guidance handler (GH), graphic simulator 
(GS), and executor (EX). NU is based upon the already developed "augmented transition network” capable of 
interpreting limited vocabulary and grammer. Typed natural-language-like commands are interpreted by NLI. Voice 
commands can be added later, n works with NLI to interpret the meaning of the operator’s mouse-cursor commands. 
GI takes input from NLI and II and determines robot motions accordingly. GH handles inputs from the 6 DOF 
joystick and also moves the robot. GS checks if commanded motions are free of collision, and if so EX is called to 
execute the motions. 


Keyboard 



Experiments. Experimental subjects are being asked to perform a telerobotic obstacle avoidance task to evaluate 
the effectiveness of the command language. 
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